Robustness Checks and Robustness Tests in Applied Economics

A common exercise in empirical studies is a "robustness check," where the researcher examines how certain "core" regression coe¢ cient estimates behave when the regression speci…cation is modi…ed by adding or removing regressors. If the coe¢ cients are plausible and robust, this is commonly interpreted as evidence of structural validity. Here, we study when and how one can infer structural validity from coe¢ cient robustness and plausibility. As we show, there are numerous pitfalls, as commonly implemented robustness checks give neither necessary nor su¢ cient evidence for structural validity. Indeed, if not conducted properly, robustness checks can be completely uninformative or entirely misleading. We discuss how critical and non-critical core variables can be properly speci…ed and how noncore variables for the comparison regression can be chosen to ensure that robustness checks are indeed structurally informative. We provide a straightforward new Hausman (1978)type test of robustness for the critical core coe¢ cients, additional diagnostics that can help explain why robustness test rejection occurs, and a new estimator, the Feasible Optimally combined GLS (FOGLeSs) estimator, that makes relatively e¢ cient use of the robustness check regressions. A new procedure for Matlab, testrob, embodies these methods.